Search for: All records

This study aims at sensing and understanding the worker’s activity in a human-centered intelligent manufacturing system. We propose a novel multi-modal approach for worker activity recognition by leveraging information from different sensors and in different modalities. Specifically, a smart armband and a visual camera are applied to capture Inertial Measurement Unit (IMU) signals and videos, respectively. For the IMU signals, we design two novel feature transform mechanisms, in both frequency and spatial domains, to assemble the captured IMU signals as images, which allow using convolutional neural networks to learn the most discriminative features. Along with the above two modalities, we propose two other modalities for the video data, i.e., at the video frame and video clip levels. Each of the four modalities returns a probability distribution on activity prediction. Then, these probability distributions are fused to output the worker activity classification result. A worker activity dataset is established, which at present contains 6 common activities in assembly tasks, i.e., grab a tool/part, hammer a nail, use a power-screwdriver, rest arms, turn a screwdriver, and use a wrench. The developed multi-modal approach is evaluated on this dataset and achieves recognition accuracies as high as 97% and 100% in the leave-one-out and half-half experiments, respectively. 

In a human-centered intelligent manufacturing system, every element is to assist the operator in achieving the optimal operational performance. The primary task of developing such a human-centered system is to accurately understand human behavior. In this paper, we propose a fog computing framework for assembly operation recognition, which brings computing power to the data source, to achieve real-time recognition. The operator’s activity is captured using visual cameras. Instead of directly training a deep learning model from scratch, transfer learning is applied to transfer the learning abilities to our application. A worker assembly operation dataset is established, which at present contains 10 sequential operations in an assembly task of installing a desktop CNC machine. The developed model is evaluated on this dataset and achieves a recognition accuracy of 95% in the testing experiments. 

Training and on-site assistance is critical to help workers master required skills, improve worker productivity, and guarantee the product quality. Traditional training methods lack worker-centered considerations that are particularly in need when workers are facing ever changing demands. In this study, we propose a worker-centered training & assistant system for intelligent manufacturing, which is featured with self-awareness and active-guidance. Multi-modal sensing techniques are applied to perceive each individual worker and a deep learning approach is developed to understand the worker’s behavior and intention. Moreover, an object detection algorithm is implemented to identify the parts/tools the worker is interacting with. Then the worker’s current state is inferred and used for quantifying and assessing the worker performance, from which the worker’s potential guidance demands are analyzed. Furthermore, onsite guidance with multi-modal augmented reality is provided actively and continuously during the operational process. Two case studies are used to demonstrate the feasibility and great potential of our proposed approach and system for applying to the manufacturing industry for frontline workers. 

The outbreak of Zika virus (ZIKV) in 2016 created worldwide health emergency which demand urgent research efforts on understanding the virus biology and developing therapeutic strategies. Here, we present a time-resolved chemical proteomic strategy to track the early-stage entry of ZIKV into host cells. ZIKV was labeled on its surface with a chemical probe, which carries a photocrosslinker to covalently link virus-interacting proteins in living cells on UV exposure at different time points, and a biotin tag for subsequent enrichment and mass spectrometric identification of the receptor or other host proteins critical for virus internalization. We identified Neural Cell Adhesion Molecule (NCAM1) as a potential ZIKV receptor and further validated it through overexpression, knockout, and inhibition of NCAM1 in Vero cells and human glioblastoma cells U-251 MG. Collectively, the strategy can serve as a universal tool to map virus entry pathways and uncover key interacting proteins.

 

Cdc14 protein phosphatases play an important role in plant infection by several fungal pathogens. This and other properties of Cdc14 enzymes make them an intriguing target for development of new antifungal crop treatments. Active site architecture and substrate specificity of Cdc14 from the model fungusSaccharomyces cerevisiae(ScCdc14) are well-defined and unique among characterized phosphatases. Cdc14 appears absent from some model plants. However, the extent of conservation of Cdc14 sequence, structure, and specificity in fungal plant pathogens is unknown. We addressed this by performing a comprehensive phylogenetic analysis of the Cdc14 family and comparing the conservation of active site structure and specificity among a sampling of plant pathogen Cdc14 homologs. We show that Cdc14 was lost in the common ancestor of angiosperm plants but is ubiquitous in ascomycete and basidiomycete fungi. The unique substrate specificity of ScCdc14 was invariant in homologs from eight diverse species of dikarya, suggesting it is conserved across the lineage. A synthetic substrate mimetic inhibited diverse fungal Cdc14 homologs with similar low µMK_ivalues, but had little effect on related phosphatases. Our results justify future exploration of Cdc14 as a broad spectrum antifungal target for plant protection.

 

Production innovations are occurring faster than ever. Manufacturing workers thus need to frequently learn new methods and skills. In fast changing, largely uncertain production systems, manufacturers with the ability to comprehend workers' behavior and assess their operation performance in near real-time will achieve better performance than peers. Action recognition can serve this purpose. Despite that human action recognition has been an active field of study in machine learning, limited work has been done for recognizing worker actions in performing manufacturing tasks that involve complex, intricate operations. Using data captured by one sensor or a single type of sensor to recognize those actions lacks reliability. The limitation can be surpassed by sensor fusion at data, feature, and decision levels. This paper presents a study that developed a multimodal sensor system and used sensor fusion methods to enhance the reliability of action recognition. One step in assembling a Bukito 3D printer, which composed of a sequence of 7 actions, was used to illustrate and assess the proposed method. Two wearable sensors namely Myo-armband captured both Inertial Measurement Unit (IMU) and electromyography (EMG) signals of assembly workers. Microsoft Kinect, a vision based sensor, simultaneously tracked predefined skeleton joints of them. The collected IMU, EMG, and skeleton data were respectively used to train five individual Convolutional Neural Network (CNN) models. Then, various fusion methods were implemented to integrate the prediction results of independent models to yield the final prediction. Reasons for achieving better performance using sensor fusion were identified from this study. 

Studying the dynamic interaction between host cells and pathogen is vital but remains technically challenging. We describe herein a time‐resolved chemical proteomics strategy enabling host and pathogen temporal interaction profiling (HAPTIP) for tracking the entry of a pathogen into the host cell. A novel multifunctional chemical proteomics probe was introduced to label living bacteria followed by in vivo crosslinking of bacteria proteins to their interacting host‐cell proteins at different time points initiated by UV for label‐free quantitative proteomics analysis. We observed over 400 specific interacting proteins crosslinked with the probe during the formation ofSalmonella‐containing vacuole (SCV). This novel chemical proteomics approach provides a temporal interaction profile of host and pathogen in high throughput and would facilitate better understanding of the infection process at the molecular level.